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I.  Summary  of  Completed  Project 

This  project  addresses  the  development  of  the  theory  and  implementation  of 
primal-dual  interior-point  algorithms  for  semidefinite  and  cone  programming  prob¬ 
lems.  More  specifically,  its  objectives  consist  in: 

1)  advancing  the  knowledge  of  the  theory  of  polynomial  and  superlinear  conver¬ 
gence  analysis  of  primal-dual  methods  for  SDP  and  cone  programming; 

2)  studying  the  existence  and  asymptotic  behavior  of  continuous  trajectories  for 
SDP; 

3)  developing  superlinearly  convergent  higher-order  primal-dual  interior-point  meth¬ 
ods  for  SDP  problems  without  strictly  complementary  solutions; 

4)  developing  interior-point  primal-dual  algorithms  to  solve  more  general  classes 
of  cone  programming  problems; 

5)  developing  new  and/or  improving  existing  algorithms  and  implementations  for 
first-order  NLP  methods  for  SDP; 

6)  implementing  these  new  methods  and  comparing  them  against  existing  SDP 
methods; 

7)  developing  and  implementing  SDP-based  heuristics  for  solving  various  NP-hard 
combinatorial  optimization  problems; 

8)  investigating  and  exploiting  the  special  structure  of  SDP  relaxations  of  combi¬ 
natorial  optimization  problems;  and 

9)  developing  specialized  SDP-based  implicit  enumeration  methods  for  certain 
combinatorial  optimization  problems. 
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During  the  duration  of  this  project,  a  total  of  fourteen  reports  [1,  2,  3,  4,  5,  6,  7,  8, 
9, 10, 11, 12, 13, 14]  have  been  written  which  acknowledge  the  grant  N00014-03-1-0401. 
The  works  [1,  3,  4,  9,  12,  13,  14]  have  appeared  or  will  appear  in  press:  [4,  12,  13,  14] 
in  SIAM  Journal  on  Optimization  and  [1,  3,  9]  in  Mathematical  Programming. 

II.  Main  Research 

Papers  [13,  14],  written  jointly  with  Takashi  Tsuchiya,  present  new  complexity 
results  for  interior-point  algorithms  for  linear  programming.  They  are  related  with 
topic  1  of  the  above  list  of  goals. 

Paper  [14]  presents  a  variant  of  Vavasis  and  Ye’s  layered-step  path  following 
primal-dual  interior-point  algorithm  for  linear  programming.  The  algorithm  is  a 
predictor-corrector  type  algorithm  which  uses  from  time  to  time  the  least  layered 
squares  (LLS)  direction  in  place  of  the  affine  scaling  direction.  It  has  the  same 
iteration-complexity  bound  of  Vavasis  and  Ye’s  algorithm,  namely  0(n3,5log(x4+n)) 
where  n  is  the  number  of  nonnegative  variables  and  Xa  is  a  certain  condition  number 
associated  with  the  constraint  matrix  A.  Vavasis  and  Ye’s  algorithm  requires  explicit 
knowledge  of  xa  (which  is  very  hard  to  compute  or  even  estimate)  in  order  to  compute 
the  layers  for  the  LLS  direction.  In  contrast,  our  algorithm  uses  the  affine  scaling 
direction  at  the  current  iterate  to  determine  the  layers  for  the  LLS  direction,  and 
hence  does  not  require  the  knowledge  of  xa ■  A  variant  with  similar  properties  and 
with  the  same  complexity  has  been  developed  by  Megiddo,  Mizuno  and  Tsuchiya. 
However,  their  algorithm  needs  to  compute  n  LLS  directions  on  every  iteration  while 
ours  computes  at  most  one  LLS  direction  on  any  given  iteration. 

Paper  [13]  presents  a  new  iteration-complexity  bound  for  the  Mizuno-Todd-Ye 
predictor-corrector  (MTY  P-C)  primal-dual  interior-point  algorithm  for  linear  pro¬ 
gramming.  The  analysis  of  the  paper  is  based  on  the  important  notion  of  crossover 
events  introduced  by  Vavasis  and  Ye.  For  a  standard  form  linear  program  min{cTx  : 
Ax  =  b,  x  >  0}  with  decision  variable  x  €  3?”,  we  show  that  the  MTY  P-C  algo¬ 
rithm  started  from  a  well-centered  interior-feasible  solution  with  duality  gap  n/i0  finds 
an  interior-feasible  solution  with  duality  gap  less  than  nr]  in  0(n2  log(log(/i0/r/))  + 
n3-5log(x^  +  n))  iterations,  where  Xa  a  scaling  invariant  condition  number  asso¬ 
ciated  with  the  matrix  A.  More  specifically,  Xa  is  the  infimum  of  all  the  conditions 
numbers  xad,  where  D  varies  over  the  set  of  positive  diagonal  matrices.  Under  the 
setting  of  the  Turing  machine  model,  our  analysis  yields  an  0(u3'5La  +  n2  log  L)) 
iteration-complexity  bound  for  the  MTY  P-C  algorithm  to  find  a  primal-dual  op¬ 
timal  solution,  where  La  and  L  are  the  input  sizes  of  the  matrix  A  and  the  data 
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(. A,b,c ),  respectively.  In  contrast,  the  classical  iteration-complexity  bound  for  the 
MTY  P-C  algorithm  depends  linearly  on  L  instead  of  log  L. 

Papers  [1]  presents  a  first-order  NLP  algorithms  for  solving  large-scale  semidef- 
inite  programming  (SDP)  problems.  The  distinguishing  feature  of  the  proposed  al¬ 
gorithm  is  a  change  of  variables  that  replaces  the  symmetric,  positive  semidefinite 
variable  X  of  an  SDP  problem  in  standard  form  with  a  rectangular  variable  ma¬ 
trix  R  according  to  the  factorization  X  =  RRT.  The  rank  of  the  factorization,  i.e., 
the  number  of  columns  of  R,  is  chosen  minimally  so  as  to  enhance  computational 
speed  while  maintaining  equivalence  with  the  SDP  problem.  Fundamental  results 
concerning  the  convergence  of  the  algorithm  are  derived,  and  exceptional  compu¬ 
tational  results  on  several  large-scale  test  SDP  problems  are  also  presented.  Our 
computational  experiments  have  shown  that  our  method  substantially  outperforms 
the  other  available  algorithms  (e.g.,  interior-point  methods  and  the  spectral  bundle 
method)  for  solving  large-scale  SDP  problems.  For  example,  our  method  can  now 
solve  MAXCUT  semidefinite  relaxations  of  graphs  containing  20,000  nodes  in  less 
than  10  minutes.  On  the  other  hand,  the  spectral  bundle  method  takes  more  than 
60  hours  to  solve  these  problems  and  the  interior-point  methods  can  not  even  per¬ 
form  the  first  iteration  due  to  the  large  size  of  these  problems.  Our  algorithm  has 
been  implemented  in  a  code  called  SDP-LR,  which  can  be  download  from  the  website 
http://www.isye. gatech.edu/~monteiro/software.  It  is  related  with  topics  5  and 
6  of  the  above  list  of  goals. 

Paper  [9]  surveys  the  most  recent  methods  that  have  been  developed  for  the  so¬ 
lution  of  semidefinite  programs.  It  first  concentrates  on  the  methods  that  have  been 
primarily  motivated  by  the  interior  point  (IP)  algorithms  for  linear  programming, 
putting  special  emphasis  in  the  class  of  primal-dual  path-following  algorithms.  It 
then  discusses  methods  that  have  been  developed  for  solving  large-scale  SDP  prob¬ 
lems.  These  include  first-order  nonlinear  programming  (NLP)  methods  and  more 
specialized  path-following  IP  methods  which  use  the  (preconditioned)  conjugate  gra¬ 
dient  or  residual  scheme  to  compute  the  Newton  direction  and  the  notion  of  matrix 
completion  to  exploit  data  sparsity.  The  P.I.  gave  a  semi-plenary  lecture  at  the  18th 
International  Symposium  on  Mathematical  Programming  about  the  material  of  paper 

[91- 

Solving  systems  of  linear  equations  with  “normal”  matrices  of  the  form  AD2AT  is 
a  key  ingredient  in  the  computation  of  search  directions  for  interior-point  algorithms. 
Paper  [12],  establishes  that  a  well-known  basis  preconditioner  for  such  systems  of 
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linear  equations  produces  scaled  matrices  with  uniformly  bounded  condition  numbers 
as  D  varies  over  the  set  of  all  positive  diagonal  matrices.  In  particular,  it  is  shown 
that  when  A  is  the  node-arc  incidence  matrix  of  a  connected  directed  graph  with  one 
of  its  rows  deleted,  then  the  condition  number  of  the  corresponding  preconditioned 
normal  matrix  is  bounded  above  by  m(n  —  m  +  1),  where  m  and  n  are  the  number  of 
nodes  and  arcs  of  the  network.  This  paper  is  related  with  topic  1  of  the  above  list  of 
goals. 

Paper  [10]  considers  a  variant  of  a  well-known  long-step  primal-dual  infeasible  IP 
algorithm  for  solving  the  linear  program  min{cTa; :  Ax  =  b,  x  >  0},  A  €  3?mxn,  where 
the  search  directions  are  computed  by  using  iterative  linear  solvers.  More  specifically, 
the  preconditioner  studied  in  [12]  is  used  to  precondition  the  normal  coefficient  matrix 
and  the  resulting  preconditioned  normal  system  of  equations  is  solved  by  a  standard 
iterative  linear  solver.  It  is  shown  that  the  number  of  (inner)  iterations  of  the  iterative 
linear  solver  at  each  (outer)  iteration  of  the  algorithm  is  bounded  by  a  polynomial  in 
m,  n  and  a  certain  condition  number  associated  with  A,  while  the  number  of  outer 
iterations  is  bounded  by  0(n2  log  e-1),  where  e  is  a  given  relative  accuracy  level.  As 
a  special  case,  it  follows  that  the  total  number  of  inner  iterations  is  polynomial  in  m 
and  n  for  the  minimum  cost  network  flow  problem.  This  paper  is  related  with  topic 
1  of  the  above  list  of  goals. 

Paper  [8]  extends  the  above  ideas  to  the  context  of  the  convex  quadratic  program 
mm{cFx  +  xTQx/2  :  Ax  =  b,  x  >  0}.  The  key  in  this  extension  is  the  introduction  of 
an  augmented  normal  equation  (ANE)  for  determining  the  primal-dual  search  direc¬ 
tions  for  the  convex  QP,  which  can  be  nicely  preconditioned  by  using  the  approach 
studied  in  [12].  An  approach  to  compute  a  suitable  inexact  primal-dual  search  direc¬ 
tion  is  proposed  and  complexity  results  similar  to  the  ones  obtained  for  LP  in  [12] 
are  also  developed. 

The  conjugate  gradient  (CG)  algorithm  is  well-known  to  have  excellent  theoretical 
properties  for  solving  linear  systems  of  equations  Ax  =  b  where  the  n  x  n  matrix 
A  is  symmetric  positive  definite.  However,  for  extremely  ill-conditioned  matrices 
the  CG  algorithm  performs  poorly  in  practice.  Paper  [11]  discusses  an  adaptive 
preconditioning  procedure  which  improves  the  performance  of  the  CG  algorithm  on 
extremely  ill-conditioned  systems.  It  introduces  the  preconditioning  procedure  by 
applying  it  first  to  the  steepest  descent  algorithm.  Then,  the  same  techniques  are 
extended  to  the  CG  algorithm,  and  convergence  to  an  e-solution  in  0(logdet(A)  + 
y^nloge-1)  iterations  is  proven,  where  det(A)  is  the  determinant  of  the  matrix. 

In  the  series  of  papers  [3,  4,  5,  7],  the  convergence  of  the  weighted  central  paths 
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associated  with  two  central  path  maps  and  the  central  path  for  a  class  of  degenerate 
semidefinite  programs  are  studied.  As  a  result,  we  obtain  a  new  error  bound  that 
might  eventually  be  useful  in  the  derivation  of  new  superlinear  convergence  results 
for  interior-point  primal-dual  semidefinite  programming  methods.  These  papers  are 
related  with  topic  2  of  the  above  list  of  goals. 

III.  Related  Research. 

Papers  [2,  6]  are  not  too  related  with  the  main  topics  of  this  project  but  have  been 
written  during  its  duration,  and  hence  acknowledge  it. 

Paper  [2]  considers  the  proximal  point  method  with  Bregman  distance  applied 
to  linear  programming  problems,  and  study  the  dual  sequence  obtained  from  the 
optimal  multipliers  of  the  linear  constraints  of  each  subproblem.  Convergence  of 
this  dual  sequence  is  established,  as  well  as  convergence  rate  results  for  the  primal 
sequence,  for  a  suitable  family  of  Bregman  distances.  These  results  are  obtained  by 
studying  first  the  limiting  behavior  of  a  certain  perturbed  dual  path  and  then  the 
behavior  of  the  dual  and  primal  paths. 

Paper  [6]  presents  a  modified  nearly  exact  (MNE)  method  for  solving  low-rank 
trust  region  (LRTR)  subproblem.  The  LRTR  subproblem  is  to  minimize  a  quadratic 
function,  whose  Hessian  is  a  positive  diagonal  matrix  plus  an  explicit  low-rank  update, 
subject  to  a  Dikin-type  ellipsoidal  constraint,  whose  scaling  matrix  is  positive  definite 
and  has  the  similar  structure  as  the  objective  Hessian  just  described.  The  nearly  exact 
(NE)  method  proposed  by  More  and  Sorensen  [15]  is  properly  modified  to  solve  the 
LRTR  subproblem  by  completely  avoiding  the  computation  of  Cholesky  factorizations 
of  large-scale  matrices.  The  resulting  MNE  method  is  quite  efficient  and  robust  to 
for  computing  NE  solutions  of  large-scale  LRTR  subproblems.  This  method  can  be 
applied  to  solve  some  large-scale  nonlinear  programming  problems. 

IV.  Miscellaneous 

During  the  duration  of  this  project,  a  total  of  thirteen  presentations  on  my  research 
work  were  given  at  major  conferences  and  universities.  Among  these  presentations, 
the  most  influential  one  was  a  semi-plenary  talk  that  the  P.I.  gave  at  the  18th  In¬ 
ternational  Symposium  on  Mathematical  Programming  in  Copenhagen  on  August  of 
2003. 

The  P.I.  has  collaborated  with  the  following  investigators  during  the  course  of 
this  grant:  Sam  Burer  (Dept,  of  Management  Sciences,  University  of  Iowa),  Joao  X. 
da  Cruz  Neto  (Universidade  Federal  do  Piaui,  Teresina,  Brazil),  Orizon  P.  Ferreira 
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(Universidade  Federal  de  Goias,  Goiania,  Brazil),  Arkadi  Nemirovski  (Technion  Uni¬ 
versity,  Israel),  and  Takashi  Tsuchiya  (Institute  of  Statistical  Mathematics,  Japan). 

The  P.I.  is  currently  serving  as  Associate  Editor  for  three  journals,  namely:  Math¬ 
ematical  Methods  of  Operations  Research  (2002-present) ,  INFORMS  Journal  on  Com¬ 
puting  (2001-present)  and  Mathematics  of  Operations  Research  (2003-present).  He  is 
also  serving  as  Chair  of  the  INFORMS  Optimization  Society  (2002-present). 

This  grant  has  partially  supported  one  Ph.D.  student,  namely  Zhaosong  Lu,  who 
is  expected  to  graduate  on  the  summer  of  2005. 

Finally,  due  to  his  widely  cited  research  work,  the  P.I.  joined  in  2004  the  list  of 
ISI  Highly  Cited  Researchers  (see  http://www.isihighlycited.com). 
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